International Journal of Medical Informatics

In this work, we demonstrate the unprecedented value of NIH's "All of Us Research Program" (AoURP) dataset in studying maternal morbidity and building predictive machine learning (ML) models across heterogeneous populations in the United States. We developed robust and data-driven preprocessing pipelines to curate a longitudinal, multi-site, multimodal, and demographically diverse pregnancy dataset (20,253 subjects; 27,525 pregnancy episodes) from AoURP data, using electronic health records (EHR) (Conditions, Labs, Measurements) and survey responses (Social Determinant of Health (SDoH)), focusing on 7 crucial maternal health adverse outcomes. After characterizing data quality, missingness, and heterogeneity, we performed statistical correlation analysis to identify risk factors. We subsequently developed XGBoost and sequential LSTM models to predict the adverse outcomes, reaching state-of-the-art performance for multiple outcomes. We conducted model interpretability post-hoc analysis to understand success points and fairness analysis to evaluate implications for socio-economic disparities. Four practicing physicians reviewed the set of statistically significant and ML model identified features to assess their clinical validity and novelty. Most features identified through either statistical correlations or ML feature importance analysis aligned with known clinical risk factors. Several features were identified that the ML models used but that are not currently used in clinical practice and may merit further clinical investigation. Fairness analysis revealed certain associations with SDoH and age highlight areas that warrant continued monitoring. Overall, we demonstrate that meaningful populational level patterns can be extracted, and high-performing machine learning models can be trained on this longitudinal, diverse, multi-site dataset. Important risk features, particularly novel ones identified, if validated, could inform new strategies for maternal care or enable development and validation of outcome-specific, clinically deployable ML models.

Objective: The goal was to integrate structured and unstructured data from the electronic health record (EHR) to identify social and behavioral determinants of health (SBDH) factors and evaluate their association with high emergency department (ED) utilization among cardiovascular disease (CVD) patients. Methods: A custom natural language processing (NLP) algorithm was developed to extract 17 SBDH domains from clinical notes, combined with ICD-10 Z codes and screening questionnaires from the EHR. The study included patients aged 18-64 with a history of CVD. Multivariable logistic regression was used to identify associations between SBDH and high ED utilization ([≥] 5 visits) after adjusting for age, sex, race, insurance class, and chronic conditions. Results: Use of only ICD-10 codes and screening questionnaires identified patients in only a subset of the 17 SBDH domains, whereas extraction from clinical notes identified patients across all domains and increased prevalence estimates for every domain. Among 4,844 patients with CVD, 526 (10.9%) were frequent ED users. After adjustment, patients with inadequate support system (OR= 2.8), opioid abuse (OR=2.47), inadequate insurance (OR=2.42), alcohol abuse (OR=2.28), medication affordability concerns (OR=2.28), unreliable transportation (OR=1.82), financial strain (OR=1.69), and depression (OR=1.42) were significantly more likely to have frequent ED utilization. Conclusion: NLP can efficiently and more comprehensively identify socioeconomic risk factors from the EHR. Substance use, financial strain, inadequate social support, and other socioeconomic challenges may be linked to increased ED visits in patients with CVD.

Background: Machine learning models leveraging electronic health records (EHRs) can support earlier detection of sepsis in intensive care units (ICUs). However, their clinical utility depends on reproducibility across institutions and patient populations. Building on a published pipeline from the Children's Hospital of Philadelphia (CHOP), this study examines how a neonatal sepsis prediction framework performs and can be adapted to a range of intensive care environments, paediatric, cardiac, and neonatal, at Great Ormond Street Hospital (GOSH). Methods: We extracted de-identified ICU EHR data from GOSH and applied feature derivation, unit harmonisation, and temporal sampling to align with the CHOP dataset used by Masino et al. (2019). Seven classifiers were first evaluated using CHOP-trained weights to characterise cross-domain behaviour and then retrained on local data to assess recoverability and site-specific adaptation. Model discrimination was summarised by AUC and F1, and learning curves were used to explore sample efficiency and bias-variance dynamics. Results: Models achieved strong discrimination on the CHOP neonatal cohort but demonstrated reduced performance when transferred to the mixed GOSH ICU population, reflecting anticipated domain and population shift. Retraining on GOSH data restored discrimination (AUC range 0.69-0.86), with Gradient Boosting (AUC 0.86 vs AUC 0.87 at CHOP) and KNN (AUC 0.80 vs AUC 0.79 at CHOP) models performing comparably to their CHOP benchmarks. DeLong's test confirmed statistically significant gains across all classifiers (p < 0.001). Conclusion: ICU cohort and baseline demographic differences between CHOP and GOSH introduced domain shift that limited direct model transfer. Elements of the original preprocessing pipeline could not be reproduced, further constraining transportability. Yet, retraining on local data restored high discrimination, showing that the modelling framework remains robust when re-estimated in new settings. These results highlight local adaptation as a practical route to recover performance and support safe, generalisable deployment of clinical prediction models in mixed clinical environments.

Objective: We explored the implementation considerations of machine learning (ML) suicide risk identification in a health system that serves primarily American Indian patients in the Southwestern United States. Previous work developed and reported the performance of suicide risk models, but traditional performance metrics do not directly reflect clinical workflows. Materials and Methods: Using EHR data collected for Emergency Department (ED) visits between 10/01/2019 and 10/02/2021, we conducted Decision Curve Analysis (DCA) to identify the standardized net benefit (SNB) of all models previously evaluated and plotted the potential workload implications of the model with the greatest SNB. Finally, we evaluated suicide screening practices during the study period to inform implementation barriers. Results: Among 9,244 patients, 139 (1.5%) had a suicide attempt or death within 90 days of an included visit. The population-specific logistic regression model had the greatest SNB compared to other approaches. When operationalized, the 95th percentile cutoff did not substantially increase clinical workloads, while the 75th percentile cutoff substantially impacted workloads. The secondary analysis on suicide screening depicted variations in screening administration over time. Discussion: While DCA identified advantages of ML suicide risk identification over current standards, the complexities of suicide-related care limited its clinical interpretability. A stepped-care framework may mitigate potential workload burdens. Training initiatives and dynamic implementation strategies will support high-quality implementation of ML suicide risk identification. Conclusion: Our analyses support the feasibility and utility of augmenting existing suicide screening practices with ML suicide risk identification, and demonstrate the importance of nuanced analyses to inform implementation considerations.

Objective: Transformer-based models for clinical prediction using longitudinal electronic health record (EHR) data are often developed with large feature sets and long patient histories under the assumption that more data improves performance. However, high-dimensional inputs and long sequences increase computational burden, potentially limiting scalability. We evaluated how feature dimensionality and sequence length affect predictive performance, calibration, risk stratification, and computational efficiency in EHR prediction. Methods: Using longitudinal EHR data from adults with mild asthma in an integrated healthcare system, we evaluated input representation design for predicting acute asthma exacerbation. Feature dimensionality was reduced using Integrated Gradients attribution scores, univariate performance-based selection, and clinically guided selection strategies. Sequence length was varied using percentile-based truncation of patient histories. Performance was assessed using discrimination, calibration, high-risk classification, threshold-based event capture, and computational efficiency across regions. Results: Models using fewer features achieved discrimination comparable to the 80-feature reference model, with AUROC values ranging from 0.843 to 0.864 versus 0.870 for the full model. Moderate sequence-length truncation reduced training time by more than 70% with minimal loss in discrimination. Although reduced-dimensional models showed attenuation of predicted risk at the upper tail, they identified similar high-risk populations and captured comparable proportions of asthma exacerbation events at clinically relevant thresholds. Conclusion: Transformer-based prediction models maintained strong performance across reduced feature sets. While dimensionality reduction modestly affected calibration at the highest risk levels, moderate sequence-length reduction substantially reduced computational burden with limited change in overall discrimination. These findings highlight trade-offs between input complexity, predictive performance, and computational efficiency.

Accurate prediction of adverse events in the intensive care unit (ICU) following cardiothoracic surgery is crucial for timely interventions, improving patient outcomes and optimizing value in healthcare. By leveraging advanced machine learning techniques, this study demonstrates the transformative potential of predictive analytics to enhance postoperative care using a dataset from the Society of Thoracic Surgeons' (STS) database and time-series intraoperative data. We developed a multi-modal late fusion approach integrating static patient variables with intraoperative time-series data, utilizing an ensemble of models including residual neural networks for static features, CNN and Bidirectional GRU for time-series processing, XGBoost, logistic regression, SVM and Gradient Boosting. Through five-fold cross-validation, our ensemble model with balanced threshold selection achieved an AUC of 0.87, sensitivity of 0.76, specificity of 0.83, PPV of 0.40, and NPV of 0.96. Preoperative heart failure, IABP insertion, cardiopulmonary bypass (CPB) duration and white blood cell (WBC) count emerged as key predictors. Training the same pipeline on each modality separately showed that the static registry variables alone reached the same AUC of 0.87, and that the intraoperative signals alone reached 0.83, so on this cohort the two modalities act as substitutes rather than complements and fusion changed the sensitivity and precision trade-off rather than the amount of separable signal. A per-signal ablation found that no individual monitoring channel was load bearing, with mean arterial pressure and central venous pressure the least substitutable. This balanced approach optimizes the trade-off between sensitivity and specificity, providing a screening tool with high negative predictive value for monitoring high-risk patients, and the modality comparison indicates that comparable performance is attainable at sites holding registry data alone.

Acute kidney injury (AKI) is a frequent complication within intensive care units, with its sudden onset often missed. This is especially important because a timely window for intervention is required as delayed detection leads to progressively worse outcomes. Existing machine learning and deep learning models have contributed to closing this gap, but their complexity, requiring hundreds to thousands of features, and lack of generalisation pose a limitation that prevents them from being integrated into clinical workflows across different electronic health-record ecosystems. This study presents a 37-feature XGBoost model trained on the MIMIC-IV dataset with 5.4% positive cases, with hyperparameters optimised via Optuna and probabilities calibrated using isotonic regression, designed for transportability across clinical settings. Validation was conducted internally using a temporal patient-level split simulating prospective deployment, training on 2008-2016 data and testing on 2017-2022 data"External validation was performed on the eICU Collaborative Research Database, a multi-centre dataset spanning 208 US hospitals, using the trained model without retraining. SHAP TreeExplainer was used to provide feature-level explainability for individual predictions. Internal testing yielded an AUROC score of 0.794 for predicting AKI onset within a 12-24 hour window. External validation produced a 0.750 AUROC without retraining. Equitable discrimination was observed across gender, age, chronic kidney disease presence, race, and AKI stages on both datasets, with a 95% internal CI of 0.789-0.799 confirming the model's estimate stability. These results suggest that clinically useful prediction systems are achievable with substantially fewer features than current models require.

Background: Sepsis is a life-threatening condition triggered by infection and associated with dysregulated immune response of the patient followed often by multiorgan dysfunction or failure. In the clinical evolution of sepsis towards organ dysfunction, early initiation of a suited therapy significantly increases patient outcomes and reduces mortality rates. Since electronic health records provide data in a machine-readable format, this process could be supported by computerized systems. Methods: We developed an interoperable, time-sensitive CDSS that able to detect systemic inflammation and the different classifications of sepsis (bacterial/viral, suspected/proven, on admission/PICU acquired) in pediatric patients based on the analysis of routine clinical data. This application is provided as part of this publication as an open demonstrator (web application), and the usability and accuracy of the CDSS is shown by a retrospective creation of sepsis outcome labels for a routine data set of 4,655 pediatric patients. As a reference standard, the patients were manually assessed by blinded clinical experts. Results: In comparison with the reference standard, the CDSS achieved sensitivity of 96.9% (95% CI: 80.9-99.6%) and specificity of 99.1% (95% CI: 95.1-99.8%). In the context of a sepsis outcome labeling for 4,655 patients, the CDSS detected 4,342 episodes of inflammation of which 1,723 were classified as sepsis. Conclusions: We demonstrated that our routine-data based CDSS is able to perform a complex sepsis detection process with high diagnostic accuracy. Such CDSS with the ability to differentiate between SIRS, sepsis on admission, suspected and proven sepsis can prospectively support clinical management, monitoring and quality management.

Lines, drains, and airway devices are commonly used in surgical and inpatient care to support treatment and patient monitoring. While these devices are essential, prolonged device retention is associated with complications such as infection, tissue injury, thrombosis, and extended hospital stay. Understanding how long devices remain in place and how duration varies across device types is important for improving patient safety and guiding device management practices. This study analyzes device duration patterns using Patient Lines, Drains, and Airways (LDA) data containing information on device type, placement time, and removal time across multiple patient encounters. Exploratory data analysis and machine learning approaches are used to examine duration distributions, distinguish short and prolonged device retention, and identify structural patterns across device categories. The findings indicate notable variation in device duration patterns across categories, with identifiable differences in device retention behavior across device groups. This study highlights the value of observational electronic health record data for characterizing device exposure patterns and provides a foundation for future data-driven device management and risk assessment.

Background: Whether benchmark performance reflects robust clinical reasoning rather than surface-level pattern recognition remains uncertain. We evaluated the robustness of state-of-the-art large language models (LLMs) on nephrology board renewal questions using "None of the other answers" (NOTA) substitution. Methods: From 210 Japanese Society of Nephrology board renewal questions (2014-2023), two nephrologists independently reviewed all items. Questions in which NOTA became the sole correct answer after replacement were included, yielding 145 validated questions. GPT-5, GPT-4o, Gemini 2.5 Pro, and Gemini 2.0 Flash were evaluated via application programming interfaces under default settings. The primary endpoint was accuracy, and paired differences were assessed using the exact two-sided McNemar test. Results: Accuracy was significantly lower after NOTA substitution for all models: GPT-4o, 66.21% to 19.31% (drop, 46.90 percentage points [pp]); GPT-5, 87.59% to 73.10% (14.48 pp); Gemini 2.0 Flash, 58.62% to 31.03% (27.59 pp); and Gemini 2.5 Pro, 86.90% to 55.86% (31.03 pp); all P < .001. GPT-5 showed the smallest decline and the highest accuracy in both versions. Conclusions: All evaluated LLMs showed a significant robustness gap after NOTA replacement. Newer models may be more robust, but multiple-choice accuracy remains an incomplete measure of clinical reasoning robustness.

Background: The Greater Manchester Care Record (GMCR) integrates health and social care data for 2.8 million individuals. While it contains maternity information, identifying individual pregnancies, including timing and outcomes, is complex due to variations in healthcare interactions and recording practices. This study develops open-source code to create a pregnancy register within the GMCR using diagnosis- and procedure-based algorithm established in other databases. Methods: 1,448 maternity SNOMED clinical codes identified pregnancy-related primary care records among women aged 14-49 years between March 2012 and August 2023. Codes were grouped by pregnancy outcome, with a hierarchical ruling approach where multiple outcomes appeared within one episode. Pregnancy start dates were derived from hierarchical coded events or imputed using biologically plausible duration based on outcome type. Internal validation compared GMCR register episodes against Hospital Episode Statistics (HES) hospital maternity admissions; external validation compared outcome rates against published UK estimates. Results: The algorithm was developed across 379,832 women, identifying 880,926 pregnancies. Internal validation showed nearly half of register deliveries had a corresponding HES record, with a positive predictive value (PPV) of 48.1%and a median date difference of 17 days (IQR 1-30). External comparison revealed live birth (927.8 vs. 995.6 per 1,000 deliveries) and stillbirth (2.12 vs. 3.9 per 1,000 births) rates close to national estimates but a notable discrepancy for miscarriage (8.00 vs. 15.3 per 100 pregnancies) and termination of pregnancy (5.36 vs. 25.1 per 1,000 women). Demographic alignment was strong for maternal age (31 vs. 29.8 years) and deprivation (IMD10 median: 3 vs. 2.6), though ethnic minority representation was expectedly higher in the GMCR (37.0% vs. 28.7%). Conclusions: This pregnancy register enriches the GMCR for maternal health research, enabling more comprehensive and regionally tailored studies to inform healthcare delivery and policy. This work represents an important first step in developing a scalable, open-source algorithmic tool to support pregnancy research across diverse populations. By making the code openly available, it can be adapted for use in federated data environments, enhancing reproducibility and collaboration.

Background: Artificial intelligence (AI) is increasingly used in healthcare, but data on its real-world use and perceptions among obstetricians and gynecologists remain limited. Objectives: To assess familiarity, use, perceptions, and expectations regarding AI among physicians practicing obstetrics and gynecology in France. Study Design: We conducted a cross-sectional survey between June 2 and July 30, 2025, mainly distributed through professional organizations or associations and social media. Participation was voluntary and self-selected. AI use was defined as any use of an AI tool in routine clinical practice, regardless of type or frequency. Comparisons were made between AI users and non-users and between junior and senior physicians. Results: Among 469 respondents, representing approximately 5% of the target population, 199 of 466 (42.7%) reported AI use in practice; three respondents were excluded from this analysis because of discordant questionnaire responses. AI use was associated with male gender and professional status, with residents more represented among users. Junior physicians reported greater AI use than senior physicians and expressed greater concern regarding medical confidentiality. Current applications primarily involved administrative tasks, medical writing, literature retrieval, and translation; anticipated future uses centered on clinical decision support and medical data interpretation. Perceptions of several AI applications differed between groups. Conclusions: AI was reported to be used by a substantial proportion of respondents, primarily for supportive and informational tasks. These findings support further evaluation of training, validation, governance, and safe implementation of AI in obstetrics and gynecology.

Documentation burden remains a persistent challenge in nursing practice, and Ambient artificial intelligence (Ambient AI) may offer new opportunities to mitigate the burden and improve nursing practice. This brief scientific communication reports survey findings on the current status of Ambient AI implementation in U.S. inpatient nursing settings and nurses' perceptions of its anticipated benefits and concerns. We conducted a cross-sectional online survey of practicing inpatient nurses recruited from nursing-focused social media communities. Descriptive analyses were used to summarize Ambient adoption, user experience, and perceived benefits and concerns. Among 61 eligible respondents, nearly 60% (n = 37) reported that Ambient AI had been piloted or implemented in nursing at their healthcare institutions, and approximately half (49.2 %, n = 30) reported having direct experience using Ambient AI in clinical practice. Most nurses with direct experience (73.3%, n = 22) described positive experience using the technology in practice. The frequently identified perceived benefits of Ambient AI include reduced documentation time and improved patient care workflow, and improved data quality. However, nurses also raised concerns about patients' acceptance of the new technology in their care, potential nursing job displacement, and loss of nursing skills due to overreliance on AI technology. Findings suggest that Ambient AI may help reduce nursing documentation burden and workload, yet implementation strategies should address nurses' concerns about its influence on nursing professions, foster trust among clinicians and patients, and support integration into everyday nursing practice.

Evidence-based medicine (EBM) concepts are difficult for medical students to grasp. We developed a Python-Streamlit web application providing interactive visualizations to enhance EBM education. Preliminary use with first year medical students demonstrated high engagement and improved conceptual understanding, supporting the feasibility of integrating interactive, web-based tools into EBM curricula.

Background: Passively collected smartphone and wearable data, combined with genomic information, may enable scalable early identification of youth who are at risk for substance use disorder. However, heterogeneous data collected in naturalistic settings limit researcher control over data quality, requiring substantial preprocessing. Data cleaning and feature engineering decisions can meaningfully influence analytic outcomes yet are often minimally described or omitted in published work. Objectives: We provide rationales and recommendations for common preprocessing challenges in passive sensing research, including establishing inclusion criteria when "outliers" may reflect true behavioral variability, analyzing differences across smartphone operating systems, handling observations collected outside protocol windows, and engineering features for machine learning applications. Methods: We illustrate principled approaches to these challenges using smartphone and Fitbit data from the Adolescent Brain Cognitive Development Study (ABCD). The sample was drawn from the available half of the year-4 follow-up (data release 5.1; N = 4,754; ages 13-14). Results: Extreme outlying observations were concentrated among low-engagement participants with few observed days. Smartphone usage among Apple users appeared under-recorded relative to Android users when compared with self-report measures. Fitbit-derived behavior differed systematically between observations collected within versus outside protocol windows. We also identified potential errors in the study's default preprocessing pipeline. Conclusion: We recommend prioritizing person-level inclusion criteria that preserve meaningful behavioral variability, examining temporal distributions rather than assuming observations are exchangeable, and thoroughly investigating and reporting data quality issues. Although illustrated using the ABCD Study, these principles broadly apply to research using smartphones, wearables, and other digital data sources.

Background: Critically ill patients with cancer and sepsis have high in-hospital mortality, but externally validated prediction models are limited. Objective: To develop and externally validate an interpretable machine learning framework using first-day intensive care data. Methods: We used MIMIC-IV version 3.1 for development and internal validation and eICU-CRD for external validation. Eligible adults had cancer, an intensive care unit stay of at least 24 hours, and met a prespecified operational sepsis definition. The prediction landmark was 24 hours after admission. The MIMIC-IV cohort included 3,729 stays (training, n = 2, 983; internal validation, n = 746). Same-admission diagnosis-derived variables were excluded, and 345 predictors were retained. Fourteen predictive models and a dummy baseline were evaluated. Frozen pipelines and training-derived thresholds were applied to eICU-CRD without refitting or recalibration. Results: Gradient boosting was selected as the primary model and achieved an internal AUROC of 0.8480 (95% CI, 0.8173-0.8755), AUPRC of 0.6984, and Brier score of 0.1346. Important predictors included Glasgow Coma Scale components, temperature, lactate dehydrogenase, age, respiratory rate, oxygen saturation, blood urea nitrogen, and serum lactate. In eICU-CRD (n = 611), gradient boosting achieved an AUROC of 0.7483 (95% CI, 0.7026-0.7919), AUPRC of 0.6243, and Brier score of 0.1709. Random forest had the highest external AUROC in secondary comparisons (0.7731). Conclusions: First-day data supported useful internal discrimination, but performance declined under locked external validation. Multicenter validation, recalibration, threshold assessment, and prospective evaluation are required before clinical implementation.

Objectives: Characterizing patient portal message content at scale can help target efforts to manage administrative work. We developed and validated a large language model (LLM) pipeline for multi-label classification of messages using an expert-derived topic taxonomy, then characterized topic distribution across a two-year corpus. Materials and Methods: We studied all medical advice request messages sent to ambulatory clinicians at an academic medical center from 2024-2025. We convened an expert panel that derived an 11-category taxonomy through a modified Delphi process. Two annotators labeled 750 randomly selected messages (Cohen kappa 0.80), holding out 500 for evaluation. The pipeline used GPT-4o-mini in a zero-shot prompt. On the held-out set, we measured micro- and macro-averaged precision, recall, and F1, and label stability across runs. We then characterized topic distribution and co-occurrence across the corpus. Results: The pipeline achieved micro- and macro-averaged F1 of 0.89 and 0.86. Labels were identical across runs for 93.6% of messages. Across 2.4 million messages, content concentrated on a few topics. The two most common topics, Problems & Management and Medications & Prescriptions, were present in 67.9% of messages, and the four most common in 93.9%. 51.7% of messages addressed multiple topics. Discussion and Conclusion: The pipeline classified patient message topics accurately and stably across millions of messages. Message content was concentrated within a small number of topics, highlighting opportunities for targeted interventions and enabling more efficient triage, routing, and patient-facing support.

The performance of an EHR-based deep learning model trained on a small sample can be improved if more data is collected. Instead of collecting more data, the model can be trained on additional data from an analogous external source. However, this risks the model learning patterns in the external data that do not generalize to the target sample. Furthermore, data use agreements often prohibit combining datasets with medical records of different sources. We consider utilizing pre-existing methods in continual learning, namely the elastic weight consolidation (EWC) loss function and variational continual learning (VCL), both of which are regularization-based methods that we use to borrow external data and incorporate parameters from a model on external data into local model training. To investigate the utility of this modeling framework, we consider two binary classification tasks: (1) predicting which children will be diagnosed with autism spectrum disorder (ASD) from medical claims up to 18 months, and (2) predicting which patients with end-stage renal disease (ESRD) will be re-hospitalized within 30 days. Target datasets were derived from Duke University's EHR warehouse, and external datasets were sourced from either NC Medicaid claims for the ASD prediction task, or the United States Renal Data System (USRDS) for the rehospitalization prediction task. For both of these tasks, borrowing models - using either the EWC loss function or VCL - performed similarly to that of a model trained only on the full external data, when the sample size of target data used to train the model was small. That is, while a model that does not borrow using our methods performed poorly in low data regimes, the borrowing model instead matched the performance of a model trained on external data even when sample size of target data was small. In addition, an analysis of model predictions showed that models with small samples are better calibrated and more functionally similar to a model trained only on external data when the sample size is small.

Caesarean section in Bangladesh reached 51.8% of deliveries in 2025, and elective caesarean, meaning caesarean before labour began, reached 31.6%. Risk models built on national household surveys are increasingly proposed for pointing audit toward places where scheduled surgery is outrunning clinical need, but they are usually validated in ways that flatter them. Using the 2025 Bangladesh Multiple Indicator Cluster Survey, we developed four models on 9,538 women (logistic regression, elastic net, random forest, gradient boosting) and ran the same procedure under three validation designs: random five-fold cross-validation; five-fold cross-validation grouped by sampling cluster; and leave-one-division-out cross-validation. We also tested transfer between the 2019 and 2025 rounds and audited subgroup calibration. No model improved on logistic regression by a margin worth acting on: the area under the receiver operating characteristic curve ranged from 0.724 to 0.736 under cluster-grouped validation, a spread of 0.012. Validation design mattered far more than the algorithm. Grouping folds by sampling cluster changed discrimination by at most 0.0004, this survey contributing a median of 3 eligible women per enumeration area. Withholding a whole division cost 0.044 to 0.060, more than 100 times as much, and still cost 0.033 to 0.056 after the strongest predictor, an outcome-derived district rate, was removed from every model. A model fitted to 2019 data lost 0.083 when applied to 2025, and the two rounds agreed only moderately on which predictors mattered (Spearman rank correlation 0.61). Calibration held in every wealth quintile, both residence categories and seven of eight divisions; Sylhet was the exception. Elective caesarean is predictable from routine survey items, but that predictability is local. Cross-validation, including cluster-aware cross-validation, does not measure what a model would do in a district it has never seen; a geographic holdout is the cheapest design that does.

Background Stroke remains one of the leading causes of mortality and long-term disability worldwide, with low- and middle-income countries bearing a disproportionate share of the global disease burden. In India, delays in risk identification, fragmented referral pathways, and limited continuity of preventive care present significant challenges, particularly in rural communities. As a frontline health worker Accredited Social Health Activists (ASHAs) are strategically positioned to support community-based stroke prevention; however, existing workflows are frequently constrained by multi-tasking, predominantly paper-based documentation and fragmented digital systems. Advances in mobile health, artificial intelligence along with digital health ecosystem provided by Ayushman Bharat Digital Mission (ABDM) provide an opportunity to strengthen community healthcare through integrated digital platforms. Objective This protocol describes the design, system architecture, and prospective evaluation framework of ASHA Assist India, an integrated AI-assisted mobile health platform intended to support community-based stroke prevention by connecting citizens, ASHA workers, Primary Health Centres (PHCs), and higher levels of healthcare facilities within a unified digital ecosystem. Methods ASHA Assist India has been designed as a modular, cloud-based digital health platform supporting standardized data collection, longitudinal health monitoring, referral management, and AI-assisted clinical decision support. The proposed system comprises four user-facing applications corresponding to citizens, ASHA workers, PHCs, and referral hospitals, integrated through a centralized backend providing authentication, secure data management, interoperability, analytics, and notification services. The AI framework includes three planned analytical modules: (i) population-level stroke risk stratification, (ii) longitudinal stroke risk prediction, and (iii) acute stroke symptom recognition. A prospective implementation study is planned to evaluate platform usability, feasibility, workflow integration, implementation outcomes, and operational performance within routine community healthcare settings. Future validation of the AI modules will be conducted using prospectively collected longitudinal datasets. Expected Impact The proposed platform aims to strengthen community-based stroke prevention by improving digital workflow integration, facilitating coordinated referral pathways, and supporting longitudinal monitoring through the existing healthcare providers at health and wellness centres like ASHA, Community Health Officers (CHOs), ANM, etc. Beyond stroke prevention, the modular architecture is intended to provide a scalable framework for future digital health programmes addressing multiple non-communicable diseases within primary healthcare systems. Publication of this protocol establishes a transparent implementation and evaluation framework that may guide future research, digital health innovation, and implementation science in resource-constrained settings.